Papers with Transfer Paradigm
How to Set the Learning Rate for Large-Scale Pre-training? (2026.findings-acl)
Copied to clipboard
| Challenge: | Optimal configuration of the learning rate (LR) is a fundamental yet formidable challenge in large-scale pre-training. |
| Approach: | They propose a Fitting Paradigm and a Transfer Paradigme to investigate fit and transfer . they propose scalability and elucidate the reasons why module-wise parameter tuning underperforms . |
| Outcome: | The proposed model reduces the search complexity by reducing the search cost by lowering the search factor. |